docs(charter): scheduled fetch as first-class intake + mailbox connection spec (HT-92) - #102

…ent (HT-92)

Three independent reviews (two Opus-tier, one Codex) attacked this PR. The
amendment's central claim survived: the daily cron genuinely already
enqueued the same reconcile job on the same topic, and the consumer reads
the stored cursor, so this removes an internal inconsistency rather than
introducing a capability. The findings below are the ones that landed.

**The repo contradicted the charter.** The amendment updated CHARTER.md and
nothing else, leaving "no daemons, no polling loops" quoted as binding
authority in four other places — including README.md, the public front door,
which directly contradicted the charter it links to. Also corrected in
src/providers/inbound-email.ts, src/providers/queue.ts, and
specs/modules/catalog.md, whose "no IMAP polling" rationale for dropping a
FreeScout feature is now void and flagged for re-deciding on its merits.

**The spec is demoted to DRAFT.** Review found three gaps in
specs/mail/mailbox-connection.md, all sitting on the mail-semantics
invariant, all of which block its own build order:

1. Self-echo suppression has no mechanism on SMTP+IMAP. preSuppressOwnSend
   pre-seeds the ledger with the send's providerMessageId — "the SAME id the
   transport will later report." Gmail's API supplies that shared id; SMTP
   submission returns no id a later IMAP FETCH will report. Without an
   answer, every agent reply is re-ingested as an inbound customer message.
   HT-49/HT-50 were this class of bug and were found live.
2. providerMessageId is undefined for IMAP. The ledger's idempotency key
   requires a stable per-transport id; the spec itself notes a UIDVALIDITY
   change makes every stored UID meaningless, so a UID-keyed ledger
   re-ingests the whole mailbox on reset.
3. InboundEmailProvider is webhook-shaped — both methods take an HTTP
   Request, which a cron-driven fetch does not have. The claims "least new
   code" and "behind the existing seam" rested on this and were wrong.

These are recorded in the spec rather than papered over. §2's spike results
and §3's provider matrix are independently verified and stand.

**Smaller corrections.** "It consumes nothing from the push payload" was
literally false — it consumes mailboxId; the payload's historyId is logged
and never acted on. Noted too that the field carries semantically opposite
values depending on trigger, which is a latent trap. The amendment was also
inserted mid-appendix, breaking a strictly chronological run; moved to the
end after HT-75.

Verified: tsc exit 0, biome exit 0, 1512 tests pass across 75 files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…(HT-92)

Six of nine actionable findings taken; three were wording preferences,
answered on the PR rather than applied. All land on the spec — the charter
amendment drew none.

Two were claims that were wrong, not merely thin:

- "Outbound needs no DNS" overclaimed. Sending through the operator's own
  server inherits whatever SPF/DKIM/DMARC that domain already has; it does
  not guarantee deliverability. A domain with no SPF or a broken selector
  delivers just as badly through Helpthread as through its owner's normal
  client. The true claim is that we add no NEW DNS burden, unlike every
  provider-webhook alternative.
- "Errors surface verbatim" was a leak. Raw IMAP/SMTP rejection text
  routinely carries hostnames, account addresses and auth detail; echoing it
  to a browser is not acceptable. Errors are now mapped to sanitized
  messages with diagnostic codes, raw text to redacted server logs only.

Four were real gaps:

- `Send test email` could manufacture a customer conversation. It now must
  address the connected mailbox itself (never an operator-typed recipient,
  so the setup screen cannot become a way to mail third parties), carry a
  correlation nonce the pipeline drops on, and be rate-limited.
- No timeouts were specified. Every IMAP/SMTP operation now carries a
  timeout derived from the remaining invocation budget, the worker stops
  starting work it cannot finish, and connections close on success, timeout
  and failure alike. Without this one hung FETCH burns the whole invocation,
  every minute, indefinitely.
- The UID cursor advances on COMMIT, not on fetch, and holds at the last
  committed UID on partial failure. Re-fetching a committed message is
  harmless because ingest is idempotent; advancing past an uncommitted one
  loses mail silently, which §2 does not permit.
- App-password lifecycle was undocumented. They die quietly — no revocation
  signal, no refresh failure — when the account password changes, an admin
  disables them, or the account enrols in Advanced Protection. Auth failure
  now maps to the same `needs_reconnect` state a dead OAuth grant produces,
  and the reconnect path differs per provider (credential re-entry, not a
  consent redirect).

Also noted: encrypting the credential at rest is not sufficient without
deny-by-default server-only authorization — the same gap already open on
mailbox_oauth_tokens, tracked in §8.

The spec remains DRAFT; these are quality fixes to a document still blocked
on its three mail-semantics questions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

I triaged these two as wording preferences and skipped them. Both are
labelled Major and both are real; one is an overclaim repeated throughout
this PR and in the surrounding discussion.

**"Scheduled fetch already ships" was misleading.** What ships is GMAIL
history reconciliation — a cursor-based history.list walk against the Gmail
API. That proves cursor-driven, cron-triggered intake is architecturally
sound and running in production, which is what the amendment rests on. It
provides nothing toward the IMAP UID-cursor fetch this spec describes: that
adapter, its cursor semantics, its UIDVALIDITY handling and its idempotency
key are all new and substantial work. The old phrasing invited reading the
whole thing as a scheduling-only change. Now stated as "the pattern is
proven," never "the transport is built."

**"Drop the Pub/Sub dependency" read as removing support.** Both documents
said drop while the charter says push stays preferred — a direct
contradiction. Pub/Sub becomes optional, not unsupported: the push adapter,
the webhook and the watch() renewal all remain fully supported for
operators who want sub-minute latency. What changes is that they stop being
mandatory setup. Corrected in the spec's build order and in the amendment's
"what it unlocks" paragraph, and the build-order item now also notes it is
Gmail-only and does nothing for IMAP.

All nine CodeRabbit findings on this PR are now addressed. Every one landed
on the spec; the amendment itself drew none.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…T-92/HT-100)

The amendment as drafted elevated scheduled fetch to a 'first-class primary
transport, not merely a reconciliation backstop.' That framing was the drafting
assistant's, not TJ's — it was explicitly named as his call and then made for
him ('That's a business call, not an engineering one, and it's yours').

On review TJ pointed out the framing is wrong at the root: there is no
system-wide default, because the operator picks the transport at setup. Verified
against specs/deploy/gmail-inbound-runbook.md Part A — Gmail push needs a GCP
project, an Internal OAuth app, a Pub/Sub topic, a Gmail publisher grant, a
service account, and an OIDC push subscription, plus two documented org-policy
landmines hit during live provisioning. Scheduled fetch needs an app password or
OAuth. The easier path wins on adoption regardless of what any document calls
primary.

So neither is designated primary. Both are fully supported intake paths and the
charter now states the tradeoff plainly — latency against setup cost — and
leaves the choice with the operator.

TJ's approved §1 positioning line is unchanged and never claimed a primary:
'push-based delivery where providers offer it, bounded scheduled fetches where
they don't — and no resident process either way.'

The §7 appendix amendment now records what he actually ratified, with his own
words, and notes that the 'first-class primary transport' framing was removed as
assistant-authored. Two stale 'primary' claims in the mailbox-connection draft
reconciled to match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>